In a speech-enabled environment, such as a home or automobile, a user may access information or control various functions using voice input. The information and functions may be personalized for a given user. In multiple user environments, it may be advantageous to identify a given speaker from among a group of speakers.